Integrating model predictive control with federated reinforcement learning for decentralized energy management of fuel cell vehicles

Abtract The optimization-based energy management strategy (EMS) enables expertise to improve the performance of fuel cell vehicles (FCVs). Ongoing efforts are mostly focused on optimizing a centralized EMS using a variety of high-computing technologies without offering appropriate scalability and modularity for the onboard powertrain components. In real-time applications, the time-accomplishment capability of EMSs is crucial; hence, decentralized EMSs with low-cost components and limited processing capability are necessary. Local units handle the computation load on a modular platform. In addition, the decentralized system’s plug-and-play functionality minimizes the total cost. This paper presents a decentralized model predictive control (D-MPC) based on the consensus-based alternating direction method of multipliers (C-ADMM) that explicitly considers the coordination of the dynamic reactions of powertrain components and future driving profiles. In addition, a decentralized learning method is proposed to seek the optimal policy for the moving horizon dimensions in the D-MPC using the federated reinforcement learning (FRL) algorithm in order to improve processing time. Due to the deployment of a fully modular system in the proposed learning technique, agents are restricted from sharing their trajectories. Using a highly dynamic module-to-module communication layer in a fully decentralized arrangement, the powertrain components utilize the multi-step method to attain the global optimum. The performance of the proposed framework is evaluated with regards to its precision, convergence speed, and scalability. The results of numerical simulation and implementation demonstrated that the proposed method is superior to the centralized and fixed-horizon MPC approaches

A decentralized multi-agent energy management strategy based on a look-ahead reinforcement learning approach

An energy management strategy (EMS) has an essential role in ameliorating the efficiency and lifetime of the powertrain components in a hybrid fuel cell vehicle (HFCV). The EMS of intelligent HFCVs is equipped with advanced data-driven techniques to efficiently distribute the power flow among the power sources, which have heterogeneous energetic characteristics. Decentralized EMSs provide higher modularity (plug and play) and reliability compared to the centralized data-driven strategies. Modularity is the specification that promotes the discovery of new components in a powertrain system without the need for reconfiguration. Hence, this article puts forward a decentralized reinforcement learning (Dec-RL) framework for designing an EMS in a heavy-duty HFCV. The studied powertrain is composed of two parallel fuel cell systems (FCSs) and a battery pack. The contribution of the suggested multi-agent approach lies in the development of a fully decentralized learning strategy composed of several connected local modules. The performance of the proposed approach is investigated through several simulations and experimental tests. The results indicate the advantage of the established Dec-RL control scheme in convergence speed and optimization criteria. © 2021 SAE International Journal of Electrified Vehicles

Comparison of decentralized ADMM optimization algorithms for power allocation in modular fuel cell vehicles

The advanced modular powertrains are envisioned as primary part of future hybrid fuel cell vehicles (FCVs). The existing papers in the literature solely cope with the hardware side of modularity, while the software side is also vital to capitalize on the total capacity of these powertrains. Driven by this motivation, this article puts forward a comparative study of two novel decentralized convex optimization frameworks based on alternating direction method of multipliers (ADMM) to solve a multi-objective power allocation strategy (PAS) problem in a modular FCV (MFCV). The MFCV in this article is composed of two fuel cell (FC) stacks and a battery pack. Despite the existing centralized strategies for such a modular system, this manuscript proposes two decentralized PASs (Dec-PASs) based on Consensus ADMM (C-ADMM) and Proximal Jacobian ADMM (PJ-ADMM) to bridge the gap regarding the appreciation of modularity in software terms. Herein, after formulating the central PAS optimization problem, the principle of utilizing such decentralized algorithms is presented in detail. Subsequently, the performance of the proposed Dec-PASs is examined through several numerical simulations as well as experiments on a developed small-scale test bench. The obtained results illustrate that decomposition into decentralized forms enables solving the complex PAS optimization problem faster and provides modularity and flexibility. Furthermore, the proposed Dec-PASs can cope with fault and malfunction and thus augment the durability and robustness of modular powertrain systems

Benchmark of proton exchange membrane fuel cell parameters extraction with metaheuristic optimization algorithms

Proton exchange membrane fuel cell (PEMFC) models are multivariate with different nonlinear elements which should be identified accurately to assure dependable modeling. Metaheuristic algorithms are perfect candidates for this purpose since they do an informed search for finding the parameters. This paper utilizes three algorithms, namely shuffled frog-leaping algorithm (SFLA), firefly optimization algorithm (FOA), and imperialist competitive algorithm (ICA) for the PEMFC model calibration. In this regard, firstly, the algorithms are employed to find the parameters of a benchmark PEMFC model by minimizing the sum of squared errors (SSE) between the measured and estimated voltage for two available case studies in the literature. After conducting 100 independent runs, the algorithms are compared in terms of the best and the worst SSEs, the variance, and standard deviation. This comparison indicates that SFLA marginally outperforms ICA and FOA regarding the best SSE in both cases while it performs 20% and twofold better than other algorithms concerning the worst SSE. Furthermore, the obtained variance and standard deviation by SFLA are much less than the other algorithms showing the precision and repeatability of this method. Finally, SFLA is used to calibrate the model for a new case study (Horizon 500-W PEMFC) with variable temperature. © 2019 Elsevier Lt

Energy management strategies for fuel cell vehicles: A comprehensive review of the latest progress in modeling, strategies, and future prospects

Fuel cell vehicles (FCVs) are considered a promising solution for reducing emissions caused by the transportation sector. An energy management strategy (EMS) is undeniably essential in increasing hydrogen economy, component lifetime, and driving range. While the existing EMSs provide a range of performance levels, they suffer from significant shortcomings in robustness, durability, and adaptability, which prohibit the FCV from reaching its full potential in the vehicle industry. After introducing the fundamental EMS problem, this review article provides a detailed description of the FCV powertrain system modeling, including typical modeling, degradation modeling, and thermal modeling, for designing an EMS. Subsequently, an in-depth analysis of various EMS evolutions, including rule-based and optimization-based, is carried out, along with a thorough review of the recent advances. Unlike similar studies, this paper mainly highlights the significance of the latest contributions, such as advanced control theories, optimization algorithms, artificial intelligence (AI), and multi-stack fuel cell systems (MFCSs). Afterward, the verification methods of EMSs are classified and summarized. Ultimately, this work illuminates future research directions and prospects from multi-disciplinary standpoints for the first time. The overarching goal of this work is to stimulate more innovative thoughts and solutions for improving the operational performance, efficiency, and safety of FCV powertrains

An online energy management strategy for a fuel cell/battery vehicle considering the driving pattern and performance drift impacts

Energy management strategy (EMS) has a profound influence over the performance of a fuel cell hybrid electric vehicle since it can maintain the energy sources in their high efficacy zones leading to efficiency and lifetime enhancement of the system. This paper puts forward an online multi-mode EMS to efficiently split the power among the components while embracing the effects of the driving conditions and performance degradation of the fuel cell system. In this regard, firstly, a self-organizing map (SOM) is trained to cluster the driving patterns. The SOM competitive layer in this work is composed of ten driving features as inputs and it classifies the driving patterns into three classes in the output. Subsequently, a three-mode fuzzy logic controller (FLC) is designed and optimized offline by the genetic algorithm for each driving pattern. Unlike the other similar works, the output membership function of the FLC is designed based on the online identification of the maximum power and efficiency of the fuel cell system which change over time. Finally, the SOM is utilized to recognize the driving mode at each sequence and accordingly activate the most suitable mode of the FLC to meet the requested power by efficient use of the energy sources. The performance of the proposed EMS has been validated by using the hardware-in-the-loop platform for several scenarios. The experimental results analyses indicate the promising performance of the suggested methodology in terms of ameliorating hydrogen economy and the fuel cell system lifetime

An Online Energy Management Strategy for a Fuel Cell/Battery Vehicle Considering the Driving Pattern and Performance Drift Impacts

Energy management strategy (EMS) has a profound influence over the performance of a fuel cell hybrid electric vehicle since it can maintain the energy sources in their high efficacy zones leading to efficiency and lifetime enhancement of the system. This paper puts forward an online multi-mode EMS to efficiently split the power among the components while embracing the effects of the driving conditions and performance degradation of the fuel cell system. In this regard, firstly, a self-organizing map (SOM) is trained to cluster the driving patterns. The SOM competitive layer in this work is composed of ten driving features as inputs and it classifies the driving patterns into three classes in the output. Subsequently, a three-mode fuzzy logic controller (FLC) is designed and optimized offline by the genetic algorithm for each driving pattern. Unlike the other similar works, the output membership function of the FLC is designed based on the online identification of the maximum power and efficiency of the fuel cell system which change over time. Finally, the SOM is utilized to recognize the driving mode at each sequence and accordingly activate the most sui